feat: setup local dev environment for opslevel-runner with kind#302
Conversation
- configure local runner startup to use the same helper image as coding-agent jobs - improve script readability so run arguments are easier to inspect and maintain - clarify test job enqueueing purpose by naming it for runner jobs
- add a short sleep command so the test job remains available long enough for enqueue behavior to be observed
initial approach didn't work
- lower the default enqueue count to make local test runs faster and less resource-intensive - pass configurable job pod CPU and memory requests so local runner scripts can match constrained environments
| faktory: faktory | ||
| runner: go run main.go run --mode=faktory --queues=runner --job-pod-max-wait=300 No newline at end of file | ||
| runner: ../bin/opslevel-runner-runner | ||
| image-builder: watchexec --watch . --exts go,mod,sum --watch ../Dockerfile -- HELPER_IMAGE=localhost/opslevel-runner:local ../bin/build-helper-image.sh opslevel-runner |
There was a problem hiding this comment.
this seems like overkill - hot reloading the image in kind whenever anything changes?
There was a problem hiding this comment.
good point, I'll look to narrow this down
|
|
||
| HELPER_IMAGE="${OPSLEVEL_JOB_POD_HELPER_IMAGE:-localhost/opslevel-runner:local}" | ||
|
|
||
| echo "Starting runner (mode=faktory queues=$FAKTORY_QUEUES queue=${OPSLEVEL_QUEUE:-<none>}) ..." |
There was a problem hiding this comment.
we're adding a bunch of machinery, but at the end of the day what we really need is a way to bypass the opslevel api and test how jobs are enqueued to k8s... we could probably just rig up a way to push work to worker queues directly, or maybe a new test mode that we can just pipe json into stdin or something
jasonopslevel
left a comment
jasonopslevel
left a comment
There was a problem hiding this comment.
i'm ok with this, in the very least it helps document and give examples of how to run it. It's still pretty dependent on the monolith being run locally for full end to end testing, but this is a good start.
2 participants
Motivation
Prior dev loop required manual cluster bring-up, manual helper image rebuilds, and manual restart on code changes to test
opselvel-runneragainst a kubernetes cluster.This PR enhances the
Taskfilewith commands to setup conveniently an easier development environment.Changes
Taskfile
start-faktory): brings upkindvia `setup-kind dep, then launches goremanbuild-helper-image,stop-kindinstall-depssplit per OS (darwin/linux); installsredis,watchexec,kindinstall-faktoryupgraded to v1.9.3, fetches asset URL via GitHub API + jq, idempotent via status: checkOPSLEVEL_GO_PKGbumped to v2026image-builder— rebuilds helper image on Go/Dockerfile changes, reloads into kindAdded
bin/for utility scriptskind-env.sh— shared sourcing helper: sources.env.localto exports KUBECONFIG or other local overrides, pickspodmanif available or fallback todockersetup-kind.sh— idempotent cluster create/start with lockfile mutex for concurrent goreman processes (multiple runningopslevel-runnerinstances;- stop-kind.sh — deletes job pods, stops control-plane containerbuild-helper-image.sh— checksums Go sources + DockerfileAdded
tests/(moved from src/scripts/)goreman)Repository structure modifications
src/Dockerfilemove to repository root Dockerfiledist/not to pollutesrc/Notes for reviewers
opslevel-runner; override via positional arg to scripts.bin/setup-kind.shuses a transient lockfile to serialize concurrent cluster creation whengoremanstarts multiple processes in parallellocalhost/opslevel-runner:localkindkubernetes cluster for testing.Testing performed
goremanprocesses start, image-builder loads helper into kindCaveats
Setup on Linux not fully supported at this time.